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Bellwether Education Partners is a national nonprofit 
focused on dramatically changing education and life 
outcomes for underserved children. We do this by helping 
education organizations accelerate their impact and 

by working to improve policy and practice. Bellwether 
envisions a world in which race, ethnicity, and income 
no longer predict opportunities for students, and the 
American education system affords all individuals the 
ability to determine their own path and lead a productive 
and fulfilling life. For more on Bellwether, please visit 
https://bellwethereducation.org. 


@® nacsa 


The National Association of Charter School Authorizers 
(NACSA) is working to double the number of students in great 
public charter schools by advancing policies and practices that 
promote quality, autonomy, and choice. As an independent 
voice for quality charter school authorizing, NACSA uses data 
and evidence to encourage smart charter school growth. 
NACSA works with authorizers and partners to create the gold 
standard for authorizing and build authorizers’ capacity to 
make informed decisions. NACSA also provides research and 
information that help policymakers and advocates move past 
the rhetoric to make evidence-based policy decisions. For more 
on NACSA, please visit www.qualitycharters.org. 


WHY AUTHORIZING MATTERS, ESPECIALLY DURING A PANDEMIC 


What does it mean to uphold the bar for quality in the midst of a pandemic? 


Quality, community-responsive authorizing creates public education opportunities for families and children 
by focusing on what works to meet the unique needs of every student. Great authorizers believe in the 
charter school promise and relentlessly pursue excellence, never backing down from their commitment to 
high standards, while working beside and with their communities and schools. 


But COVID-19 has significantly shifted the way schooling looks. 


School systems around the country found themselves in unprecedented predicaments during the 

2019-20 school year. Many states canceled end-of-year assessments and hit pause on school performance 
scores and ratings. These disruptions have contributed to a wide variety of data gaps, leaving authorizers 

in uncharted waters as they navigate how best to assess schools on their obligation to provide an excellent 
education to every student. 


The importance of robust, comprehensive assessments cannot be overstated. Quality assessments provide 
educators with important information on what content students have mastered and what remains 

a challenge. And when used correctly, students’ performance, and more importantly, their growth, on 

such tests can shed light on how a school is meeting the needs of their students. 


These challenging times also present an opportunity for authorizers to advance accountability beyond state 
assessments. Test results have never provided a complete picture. A school’s goal should not just be to 
help a child survive post-graduation, but to thrive. That level of quality has to be measured more broadly. 


If there was ever a time for authorizers to reflect on and consider changes to how they assess student 
learning, school performance, and quality, it is now. 
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HOW TO USE THIS GUIDE AND THE CORRESPONDING TOOLS 


In Spring 2020, NACSA convened a working group of authorizers and other stakeholders to learn more about 
the challenges they faced, given the data gaps caused by COVID-19, and to ask what guidance was needed 
for the sector to adapt to these unusual circumstances. 


Based on their feedback and insights, NACSA created this guide and accompanying resources (linked below) 
to support authorizers as they reflect on these challenges and opportunities. Authorizers are invited to: 


1 | READ this guide to gain context and spur thinking on what it means right now for authorizers 
to maintain high standards of accountability, regardless of the data gaps. 


2 | COMPLETE—with students, families, and educators—the Authorizer Worksheet (Tab 2 in our 
Workbook). The top row of Tab 2 provides key questions for authorizers to reflect on as they 
develop their approach to assessing student success and wellness. The rest of the spreadsheet 
is blank so authorizers can record their answers. The tabs that follow provide sample metrics 
across indicators. These metrics are not meant to be comprehensive. Authorizers should work 
with their schools and communities to determine if there are additional indicators and measures 
to consider. 


GATHER ideas from the Exploring Alternative Measures of Student Success and Wellness 
Toolkit. The Toolkit provides examples of how an authorizer might complete the Authorizer 
Worksheet, though only for one sample metric across each indicator. 


3 


Just as the pandemic has proven to be ever-changing, these resources are dynamic documents that may be 
updated as circumstances shift. 
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PRIORITIES, BASED ON OUR PRINCIPLES 


Any work to hold schools to high standards should be guided by these priorities, which reflect our principles: 


= Assess to ensure access: Charter schools should be accessible to all students. Assessing school 
quality plays a crucial role in ensuring equity and expanding access to a great education. 


= Serve all equitably: We cannot address what we do not see. Data from diagnostic and/or baseline 
assessments spotlight systemic inequities and help leaders identify students and schools that need 
extra support so authorizers can ensure there are plans to provide those supports. 


= Protect student interests and reduce compliance burdens on educators: These are both key 
responsibilities of authorizers. 


= Measure more than academics: Authorizers should understand how well their portfolios of schools 
are serving students, looking at a more complete picture of school quality, beyond students’ academic 
achievement. 


= Listen and respond to communities: Community-responsive authorizing means authorizers listen to 
what students, families, and educators need and act boldly, even if that means doing things outside of 
traditional authorizing activities. 


= Build on knowledge: Approaches to addressing current challenges should build on the work that 
authorizers have already been doing to create new ways to assess and communicate school quality and 
performance. 


= Adapt: Authorizers are always adapting to new circumstances, including those of the current pandemic, 
which are likely to continue well into the 2020-21 school year and affect key authorizing decisions for 
years to come. 


= Use opportunities to innovate: Adaptations should not narrowly focus on addressing current data gaps. 
COVID-19 forces a faster timeline for the evolution of accountability already underway. The strategies 
authorizers adopt now can help build the foundation to better assess what we value about school 
performance going forward. 


BELLWETHER @|/NaCSa 


EDUCATIONF PARTNERS 


BELLWETHER @(|NaCSa 


EDUCATION 


ADDRESSING DATA GAPS DURING A PANDEMIC 


REFLECTING AND ACTING ON ASSESSING SCHOOL PERFORMANCE 


To identify meaningful ways to understand how well charter schools are serving students now, authorizers 
should first look carefully at how they have previously assessed school performance, the questions they 
are trying to answer with existing measures, and why those questions matter. This process cannot be done 
in isolation: to be done well, schools and families should be engaged early and often. 


NACSA encourages authorizers to explore and answer, with both internal and external stakeholders: 


1.| What are the key concepts/indicators in our current/past approach to assessing school quality? Are 
there other concepts/indicators related to school performance that become newly or more important to 
assess in a COVID-19 world? 


2 | How have we measured school performance within these indicators? 
3 | Why do we focus on these measures and metrics? 
4| What questions are we trying to answer? 


5 | What kinds of decisions does this data inform? What are the stakes of these decisions for students, 
schools, communities, and authorizers? 


6 | What problems or complications may prevent us from using those measures now? 
7 | Are there other limitations of those measures that we were already concerned about or aware of? 


8 | What are other ways to get at the questions these measures are intended to help us answer? What 
information are we currently collecting from schools that might address these questions? Is there 
additional information that schools are collecting that might help address these questions? 


9 | Are there other information sources that would be useful to address these questions that authorizers 
and schools don’t currently have, but potentially could? What are the barriers (including policy barriers) 
to using these other ways of assessing quality, or using them well? How and to what extent can we 
overcome these barriers? 


WHAT ABOUT GOVERNANCE/OPERATIONS AND FINANCE? 


Charter school accountability is often talked about as a three-legged stool: authorizers are asked 
to consider 1) financial performance; 2) operations/governance performance; and 3) academic/ 
student performance. This document and the accompanying resources are focused on academic 
and student performance. Authorizers, charter school board members, and school leaders should 
work together to also determine how best to reflect on and to assess financial, operational, and 
governance health as well. 
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KEY INDICATORS FOR AUTHORIZERS TO ASSESS QUALITY 
Since authorizers’ contexts differ, each authorizer should reflect on and analyze the key indicators and 


metrics in their current/past approach to assessing school quality. NACSA advocates for authorizers to: 


= review student progress over time, student achievement, post-secondary readiness, and mission-specific 
academic goals as part of charters’ overall academic performance; 


= when possible, disaggregate data within each indicator and metric so that subgroup performance is 
evident; 


= consider additional indicators and metrics, including indicators of well-being and indicators related to 
closing opportunity gaps. 


The indicators below may be helpful starting places for many authorizers. For a more detailed, though not 
exhaustive, list of potential metrics for each indicators, please see the Workbook. 


INDICATOR METRIC EXAMPLE 


Student Growth and Progress Disaggregated data for student growth percentiles 


Disaggregated data for proficiency rates relative 


SUSIE SCTE to other schools or state average 


Postsecondary Readiness Disaggregated data for graduation rates 


Extent to which school is implementing and 
students are completing certain practices 
aligned with school mission (e.g. students 
completing internship or Capstone) 


Mission- and Model-specific Goals 


Student and Family Engagement and Well-being Disaggregated data for school climate surveys 


Opportunity Gaps Teacher-student race matching data 


NACSA is not suggesting that authorizers will or should evaluate every one of these indicators in the ways 
listed; these are simply examples. However, authorizers should review their own performance frameworks to 
determine how they compare and to reflect on the nine guiding questions above. 


Further, since authorizers do not typically have the authority to unilaterally change performance expectations 
(nor should they), authorizers and school operators need to work together on contractual changes that may 
arise when considering indicator and metric changes. 
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AN OPPORTUNITY TO ADAPT 


The pandemic has disrupted education as we know it. It is unclear when, or if, things can go back to 
“normal”—or even if they should. Authorizers must adapt to fulfill their fundamental responsibility: to hold 
charter schools accountable for meeting the needs of students and families. 


Authorizers must look carefully at how they have previously assessed school performance and understand 
the how and the why of what they have traditionally assessed. Only then can they adequately determine 
how to alternatively address data gaps. 


Now is the time for authorizers to reflect on the measures that may help capture a more robust and 
comprehensive understanding of schools’ performance and consider changes. 


We hope these resources will be useful to the field as authorizers determine how best to uphold the bar 
for quality for students and families. 


[4 VIEW THE TOOLKIT 


4 VIEW THE WORKBOOK 


POLICY CONSIDERATIONS 


Authorizers typically see themselves as practitioners and not necessarily as advocates. However, 
during these unprecedented times, we strongly encourage authorizers to be vocal with policymakers 
about their struggles and their needs. 


In some states, charter performance frameworks are tied directly to the state accountability system. 
Since accountability decisions, such as renewals, are often based on performance frameworks, 
it leaves little to no autonomy for the authorizer to make accountability decisions based on data 
outside of what is explicitly in law or policy when facing data gaps or when state accountability is 
paused. In situations like this, authorizers and their external partners could consider making the 
case to their policymakers for the authority to approach accountability in a different way. 
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